The topic of data visualization is still very popular in the data science community. The market size for visualization products is valued at $4 Billion and is projected to reach $7 Billion by the end of 2022 according to Mordor Intelligence. While we have seen amazing advances in the technology to display information, the understanding of how, why, and when to use visualization techniques has not kept up. Unfortunately, people are often taught how to make a chart before even thinking about whether or not it’s appropriate.
This is the first post of many on how to utilize data visualization techniques effectively. I will focus primarily on use cases in R but visualizations go well beyond charts and graphs.
Data Visualization Lessons, What to Keep:
I have to give credit to Junk Charts - it inspired a lot of this post.
Ask yourself the following questions to help drive your decision:
Once you know which question you are asking, it will keep your mind focused on the outcome and will quickly narrow down your charting options.
Obviously, there are choices beyond these and you need to think through your choice wisely.
Side Note: I hate donut and pie charts! When used properly, they’re terriffic! However, I’m very used to gagging every time one appears on a projector screen due to how frequently they’re used inappropriately.
For this project, I’ll use some oil production data that I found while digging through http://data.world (pretty great site). The data can be found here
#Custom data preparation
#GitHub (linked to at bottom of this post)
source('data_preparation.R')
data = getData()
head(data)
## Location Month Year ThousandBarrel Date
## 1 Alabama Mar 2013 883 2013-03-01
## 2 Alabama Apr 2013 844 2013-04-01
## 3 Alabama May 2013 878 2013-05-01
## 4 Alabama Feb 2013 809 2013-02-01
## 5 Alabama Mar 1982 1687 1982-03-01
## 6 Alabama Apr 1982 1567 1982-04-01
Objective: See what the oil production in the US looked like from 1981 - 2016 by year. I want to illustrate the changes over the time period. This is a very high-level view and only shows us a decline followed by a ramp up at the end of the period.
I decided to use a line chart to show the trend over time. When using discrete data you should use a column chart to avoid any confusion that in between these years the data actually was simply linear. However, it paints a much clearer picture this way and is not misleading.
The x-axis is a disaster and the y-axis isn’t formatted well. While it gets the point across, it’s still almost worthless.
df = data %>%
group_by(Year) %>%
summarise(ThousandBarrel = sum(ThousandBarrel))
p = ggplot(df,aes(x=Year,y=ThousandBarrel,group=1))
p + geom_line(stat='identity') +
ggtitle('Oil Production Over Time') +
theme(plot.title = element_text(hjust = 0.5),plot.subtitle = element_text(hjust = 0.5)) +
xlab('') + ylab('')
The title gives us a much better understanding of what we’re looking at. The chart is slightly wider and the axes are formatted to be legible.
p = ggplot(df,aes(x=Year,y=ThousandBarrel,group=1))
p + geom_line(stat='identity') +
ggtitle('Thousand Barrel Oil Production By Year in the U.S.') +
theme(plot.title = element_text(hjust = 0.5),plot.subtitle = element_text(hjust = 0.5)) +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
scale_y_continuous(labels = comma)
Objective: Identify which states affected the trend the most. Evaluate them simultaneously in order to paint the picture and compare them.
From this visual you can see the top states are Alaska, California, Louisiana, Oklahoma, Texas and Wyoming. Texas seems to break the mold quite drastically and drove the spike which occurred after 2010.
There are far too many colors going on here. Everything at the bottom of the chart is relatively useless and takes our focus away from the big players.
df = data %>%
group_by(Location, Year) %>%
summarise(ThousandBarrel = sum(ThousandBarrel))
df$Year = as.numeric(df$Year)
p = ggplot(df,aes(x=Year,y=ThousandBarrel,col=Location))
p + geom_line(stat='identity') +
ggtitle(paste('Oil Production By Year By State in the U.S.')) +
theme(plot.title = element_text(hjust = 0.5)) +
theme(axis.text.x = element_text(angle = 90, hjust = 1))
This focuses attention on the top producing states. It compares them to each other and shows the trend per state as well.
n=6 #Arbitrary at first, after trying a few, this made the most sense
topN = data %>%
group_by(Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel)) %>%
arrange(-ThousandBarrel) %>%
top_n(n)
df = data %>%
filter(Location %in% topN$Location) %>%
group_by(Year,Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel))
df$Year = as.numeric(df$Year)
df$Location = as.factor(df$Location)
p = ggplot(df,aes(x=Year,y=ThousandBarrel,group=1))
p + geom_line(stat='identity') +
ggtitle(paste('Top',as.character(n),'States - Oil Production By Year in the U.S.')) +
theme(plot.title = element_text(hjust = 0.5)) +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
facet_wrap(~Location) +
scale_y_continuous(labels = comma)
Objective: See if Alaska and California data is correlated (This probably isn’t important but it allows us to use the same data).
Lots of completely irrelevant data! Size of the point should have nothing to do with the year.
statesList = c('Alaska','California')
df = data %>%
filter(Location %in% statesList) %>%
spread(Location,ThousandBarrel) %>%
select(Alaska,California,Month,Year)
p = ggplot(df,aes(x=Alaska,y=California,col=Month,size=Year))
p + geom_point() +
scale_y_continuous(labels = comma) +
scale_x_continuous(labels = comma) +
ggtitle('Oil Production - CA vs. AK') +
theme(plot.title = element_text(hjust = 0.5))
The trend line is nice because it helps to visualize the relationship even more. While it can sometimes be misleading, it makes sense with our current data.
df = data %>%
filter(Location %in% statesList) %>%
spread(Location,ThousandBarrel) %>%
select(Alaska,California,Year)
p = ggplot(df,aes(x=Alaska,y=California))
p + geom_point() +
scale_y_continuous(labels = comma) +
scale_x_continuous(labels = comma) +
ggtitle('Monthly Thousand Barrel Oil Production 1981-2016 CA vs. AK') +
theme(plot.title = element_text(hjust = 0.5)) +
geom_smooth(method='lm')
Objective: Examine the range of production by state and year over the time period to give us an idea of the variance.
df = data %>%
group_by(Year,Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel))
p = ggplot(df,aes(x=Location,y=ThousandBarrel))
p + geom_boxplot() +
ggtitle('Distribution of Oil Production by State')
This gives a nice ranking to the plot while still showing their distributions. While it was semi-apparent in the line charts, the variance of Texas is huge compared to the others! We could take this a step further and separate out the big players from the smaller players.
p = ggplot(df,aes(x=reorder(Location,ThousandBarrel),y=ThousandBarrel))
p + geom_boxplot() +
scale_y_continuous(labels = comma) +
ggtitle('Distribution of Annual Oil Production By State (1981 - 2016)') +
coord_flip()
Objective: Check out the composition of total production by state.
My favorite, the beautiful pie chart! There’s nothing better than this…
df = data %>%
group_by(Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel)) %>%
mutate(ThousandBarrel = ThousandBarrel/sum(ThousandBarrel))
df$ThousandBarrel = round(100*df$ThousandBarrel,0)
library(plotrix)
pie(x=df$ThousandBarrel,labels=df$Location,explode=0.1,col=rainbow(nrow(df)),main='Percentage of Oil Production by State')
The 1980’s and 2010’s will be missing years in terms of a “decade” due to the data provided (and it’s only 2017). While the percentage labels are slightly off center, it’s certainly much better than the pie chart. It’s not quite “apples-to-apples” for a comparison because I created different decades, but you get the idea.
I also created an “Other” category in order to simplify the output. When you are doing comparisons, it’s typically a good idea to find a way to reduce the number of variables in the output while not removing data by dropping it completely.
data$Decade = '1980s'
data$Decade[data$Year >= 1990] = '1990s'
data$Decade[data$Year >= 2000] = '2000s'
data$Decade[data$Year >= 2010] = '2010s'
data$Decade = as.factor(data$Decade)
top5 = data %>%
group_by(Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel)) %>%
arrange(-ThousandBarrel) %>%
top_n(5) %>%
select(Location)
top5List = top5$Location
data$State = "Other"
for(i in 1:length(top5List)){
data$State[data$Location == top5List[i]] = top5List[i]
}
df = data %>%
group_by(Decade,State) %>%
summarise(ThousandBarrel = sum(ThousandBarrel)) %>%
mutate(ThousandBarrel = ThousandBarrel/sum(ThousandBarrel))
df$ThousandBarrel = round(df$ThousandBarrel,3)
df$text = paste(round(100*df$ThousandBarrel,0),'%', sep='')
p = ggplot(df,aes(x=Decade,y=ThousandBarrel,col=reorder(State,ThousandBarrel),fill=reorder(State,ThousandBarrel)))
p + geom_bar(stat='identity') +
geom_text(aes(label=text),col='Black',size = 4, hjust = 0.5, vjust = 3, position = "stack") +
scale_y_continuous(labels = percent) +
ggtitle('Percentage of Top Oil Producing States by Decade') +
guides(fill=guide_legend(title='State'),col=guide_legend(title='State')) +
theme(plot.title = element_text(hjust = 0.5))
Some of them are nice, others are terrible! I won’t comment on any of them, but I felt it was necessary to include some other ideas I toyed around with.
Have fun with your data visualizations. The charts I showed here are extremely simple. Being creative by using things other than R wind up making visuals people can remember. There are plenty of examples around, but they all tend to follow basic principles of design. There are A TON of good books out there on this topic.
Now it’s your turn!
df = data %>%
group_by(Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel)) %>%
arrange(-ThousandBarrel)
p = ggplot(df,aes(x=reorder(Location,ThousandBarrel),y=ThousandBarrel))
p + geom_bar(stat='identity') +
ggtitle('Oil Production 1981 - 2016 By Location') +
theme(plot.title = element_text(hjust = 0.5)) +
coord_flip()
top10 = data %>%
group_by(Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel)) %>%
arrange(-ThousandBarrel) %>%
top_n(10)
## Selecting by ThousandBarrel
print(top10)
## # A tibble: 10 × 2
## Location ThousandBarrel
## <chr> <dbl>
## 1 Texas 23447172
## 2 Alaska 15775279
## 3 California 9988225
## 4 Louisiana 4267246
## 5 Oklahoma 3701224
## 6 Wyoming 2894624
## 7 Kansas 1708873
## 8 Colorado 1288643
## 9 Utah 894657
## 10 Mississippi 861999
df = data %>%
group_by(Location,Year) %>%
filter(Location %in% top10$Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel))
p = ggplot(df,aes(x=Year,y=ThousandBarrel,col=Location,fill=Location))
p + geom_bar(stat='identity') +
ggtitle('Oil Production - Top 10 States') +
theme(plot.title = element_text(hjust = 0.5)) +
theme(axis.text.x = element_text(angle = 90, hjust = 1))
df = data %>%
filter(Year == 1990)%>%
group_by(Location) %>%
summarise(ThousandBarrel = sum(ThousandBarrel))
df$Location = tolower(df$Location)
#Add States without data
States = data.frame(Location = tolower(as.character(state.name)))
missingStates = States$Location[!(States$Location %in% df$Location)]
appendData = data.frame(Location=missingStates,ThousandBarrel=0)
df = rbind(df,appendData)
states_map <- map_data("state")
ggplot(df, aes(map_id = Location)) +
geom_map(aes(fill=ThousandBarrel), map = states_map) +
expand_limits(x = states_map$long, y = states_map$lat)
df = data %>%
filter(Location == 'Texas') %>%
group_by(Year,Month) %>%
summarise(ThousandBarrel = sum(ThousandBarrel))
p = ggplot(df,aes(x=Month,y=ThousandBarrel))
p + geom_line(stat='identity',aes(group=Year,col=Year)) +
ggtitle('Oil Production By Year in the U.S.') +
theme(plot.title = element_text(hjust = 0.5)) +
theme(axis.text.x = element_text(angle = 90, hjust = 1))
As always, the code used in this post is on my GitHub